ECE 587 / STA 563 : Lecture 5 – Lossless Compression

ECE 587 / STA 563 : Lecture 8 – Gaussian Channel

ECE 587 / STA 563 : Lecture 6 – Channel Coding

6.

ECE 587 / STA 563 : Lecture 2 – Measures of Information

2.1 Quantifying Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2.2 Entropy and Mutual Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.2.1 Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.2.2 Mutual Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2.3 Example: Test...

STA 563 : Lecture 9 – Rate Distortion Information Theory

9.1 Motivation: Quantization of Random Variables . . . . . . . . . . . . . . . . . . . . . 1 9.2 Lossy Source Coding Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 9.3 The Rate Distortion Coding Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 4 9.3.1 Example: Binary Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 9.3.2 Example: Gaussian ...

Lecture 1 : Data Compression and Entropy

In this lecture, we will study a simple model for data compression. The compression algorithms will be constrained to be “lossless” meaning that there should be a corresponding decoding algorithm that recovers the original data exactly. We will study the limits of such compression, which ties to the notion of entropy. We will also study a simple algorithm for compression when the input text arr...